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Research suggests that using only standardized placement test scores to determine 
which new college students should take developmental coursework is inadequate, 
as many students may be unnecessarily assigned to developmental courses. These 
courses generally take one or more semesters to complete, delaying entry into college- 
level coursework, and students earn no credit toward a credential for completing 
them. MDRC and the Community College Research Center recently completed two 
experimental studies of multiple measures assessment (MMA), in which colleges use 
high school GPA and other measures in addition to placement test scores to assign 
incoming students to either developmental or college-level courses in math and English. 
Overall, results from both studies indicate that student outcomes improve under MMA 
as compared to status quo placement based on test scores alone. In this brief, we 
explain what we have learned from these studies. 


The Studies 


The first study, funded by the Institute of Education Sciences through the Center for the 
Analysis of Postsecondary Readiness, was conducted at seven community colleges from the 
State University of New York (SUNY) system. The second, funded by Ascendium Education 
Group, took place at four Minnesota community colleges and one Wisconsin community 
college. The aim of both studies was to learn whether MMA yields placement determinations 
that lead to better student outcomes than a system based solely on placement test scores. 
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Both studies employed randomized controlled trial designs to determine the causal impact of 
MMA on student outcomes. Incoming students were randomly assigned to one of two groups: One 
received MMA placement and the other received the existing test-based placement. Students in 
both studies were tracked for at least three terms. The main difference between the two studies 
was the type of MMA model implemented at the participating colleges. 


In New York, SUNY colleges used algorithms (one for math and one for English) that took 
several factors into consideration before placing students into developmental or college-level 
courses: performance on a standardized test, high school GPA, and years since high school 
graduation. Each college used data on their former students to develop the algorithms by 
weighting the measures according to how well they predicted success in college-level math 
and English courses. Faculty at each college then chose cutoff points on these predictive 
algorithms to assign students to either developmental or college-level math and English 
courses. The algorithms “bumped up” some students into college-level courses who would have 
been assigned to developmental courses under the status quo system; it also “bumped down” a 
smaller number of students from college-level courses into developmental courses. 


In Minnesota and Wisconsin, participating colleges developed a set of decision rules in which 
each of several measures were considered in a specific order to determine which courses 
students were eligible to take. Typically, colleges first considered waivers to identify students 
who would be exempt from consideration of other measures. Then results from placement tests, 
high school GPA, and noncognitive assessments were considered. In some cases, a system of 
“decision bands” was used so that students who earned test scores within a certain range would 
be evaluated using other measures. Unlike in the SUNY study, the colleges in the Minnesota and 
Wisconsin study set the rules so that no student received a lower placement than they otherwise 
would have under the status quo system (i.e., they could never be “bumped down’). 


Major Takeaways 


Students bumped up by MMA are more likely to complete college-level courses. Results from 
both studies indicate that student outcomes improve under MMA compared to the status quo. 
All subgroups defined by race/ethnicity, gender, and Pell status were more likely to enroll in these 
courses. Most subgroups also saw similar benefits on completion of those courses, although 
findings for men and students of color differed slightly across subjects and studies. The SUNY 
study found significant positive effects on college-level English course completion among men 
and students of color but no significant effects on college-level math course completion. The 
Minnesota and Wisconsin study found significant positive effects in both subjects among all 
demographic subgroups. 


While most students’ placement levels were not changed by MMA, some students in both 
studies were bumped up into college-level courses. In the SUNY study, 16% of students were 
bumped up in math and 23% were bumped up in English. In the Minnesota and Wisconsin study, 
about 15% of students were bumped up in each subject. The proportions bumped up in both 
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studies were determined by faculty-set cutoff scores on the algorithm (SUNY) or on specific 
measures (Minnesota and Wisconsin). 


Students bumped up by MMA benefitted from being placed directly into college-level math and 
English courses instead of into developmental courses. Being bumped up by MMA increased 
enrollment and completion of college-level courses by roughly 10 percentage points in both 
studies and both subjects, even after three semesters, enough time for students placed into 
developmental education to have caught up. 


College-Level Course Completion Rates Among Students in the Bump-Up Zone 
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NOTE: SUNY rates include the completion of any college-level course in math or English; MN/WI rates include the 
completion of only introductory courses in these subjects. ***p < .01 


Students bumped down by MMA are less likely to complete college-level courses in math and 
English. In the SUNY study, the placement algorithm bumped down a small number of students 
into a developmental course. In those cases, MMA hurt student outcomes: Students bumped 
down to developmental courses were less likely to complete introductory college-level courses, 
even after three semesters. 


High school cumulative GPA is the best observable predictor of success in college-level 
courses. Both studies analyzed historical data from the participating colleges to understand 
the relationship between available measures of college readiness and student performance in 
college-level courses. Both studies found high school GPA to be far and away the best predictor 
of success in college-level courses, much better than placement test scores. Other measures — 
including scores from noncognitive assessments that were used in the placement of students 
in Minnesota and Wisconsin (the Learning and Study Strategies Inventory [LASSI] and the Grit 
Scale) — improved the predictive utility of the models very little if at all. 


Simpler rules are less costly to implement and just as effective. The MMA system used in the 
SUNY colleges was much more costly to implement than the MMA system based on simple 
decision rules used in the Minnesota and Wisconsin colleges, in large part because of the 
additional level of effort associated with generating placements based on weighted algorithms 
at each SUNY college. Since the studies produced similar impacts, the simpler rules appear to be 
more cost effective per credit earned. 
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Recommendations 


Taken together the two studies suggest that institutions seeking to reform placement should: 


* Use MMA to bump more students up from developmental prerequisite courses to 
introductory college-level courses. Results from both studies suggest that increased 
access to college-level courses improves students’ chances of completing college-level 
math and English courses. Evidence does not suggest that students benefit when MMA 


bumps them down from college-level to developmental 
courses. 


* Develop an MMA system that places increased emphasis 
on high school transcripts. Both studies found that high 
school GPA is the best observable measure for predicting 
success in college-level courses, while other measures, 
such as noncognitive assessment scores, add little 
predictive utility. 


* Design and implement a simple MMA system based 
on decision rules with cutoffs on a few measures. 
Comparing results of the two experimental studies 
shows that a simpler placement system works as well as 
a more complicated algorithmic system, and can be less 
expensive to implement. 


FOR MORE INFORMATION ON 


The New York study, visit: https:// 
postsecondaryreadiness.org/ 
multiple-measures-impact- 
findings/ 


The Minnesota and Wisconsin 
study, visit: https://mdrc.org/ 
publication/increasing-gatekeeper- 
course-completion 


Rigorous study of MMA at both sets of sites will continue. Future research will estimate the impact 
of MMA on student graduation rates; explore variation in impact by race/ethnicity, gender, and Pell 
subgroups; and consider the potential effects of modifying multiple measure cutoff scores. 
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